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Abstract. The paper studies closed queueing networks contain- 
ing a server station and k client stations. The server station is 
an infinite server queueing system, and client stations are single- 
server queueing systems with autonomous service, i.e. every client 
station serves customers (units) only at random instants generated 
by a strictly stationary and ergodic sequence of random variables. 
The total number of units in the network is N. The expected 
times between departures in client stations are (A/Xj) _1 . After 
a service completion in the server station, a unit is transmitted 
to the jth client station with probability pj (j = 1,2, ... ,k), and 
being processed in the jth client station, the unit returns to the 
server station. The network is assumed to be in a semi-Markov 
environment. A semi-Markov environment is defined by a finite or 
countable infinite Markov chain and by sequences of independent 
and identically distributed random variables. Then the routing 
probabilities pj (J = 1, 2, . . . , k) and transmission rates (which are 
expressed via parameters of the network) depend on a Markov state 
of the environment. The paper studies the queue- length processes 
in client stations of this network and is aimed to the analysis of 
performance measures associated with this network. The ques- 
tions risen in this paper have immediate relation to quality control 
of complex telecommunication networks, and the obtained results 
are expected to lead to the solutions to many practical problems 
of this area of research. 
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1. Introduction 



We consider closed queueing networks containing a server station 
and k client stations. The server station is an infinite server queueing 
system with identical servers. Client stations are single-server queue- 
ing systems with an autonomous service mechanism, where customers 
(units) are served only at random instants generated by a strictly sta- 
tionary and ergodic sequence of random variables. 

Queueing systems with an autonomous service mechanism were in- 
troduced and originally studied by Borovkov [TT] . [T2] . The formal defi- 
nition of these systems in the simplest case of single arrivals and depar- 
tures is as follows. Let A(t) denote an arrival point process, let S(t) de- 
note a departure point process, and let Q(t) be a queue- length process, 
and all of these processes are started at zero (A(0) = S(0) = Q(0) = 0). 
Then the autonomous service mechanism is defined by the equation: 



The queueing systems with an autonomous service mechanism have 
been studied in many papers (e.g. Abramov pQ, [3], [5], Fricker [16] . 
[T7| . Gelenbe and Iasnogorodski [18]). The structure of queueing sys- 
tems or networks with autonomous service and their analysis is much 
easier than those structure and analysis of "usual" systems with gen- 
erally distributed service times. Queueing systems and networks with 
autonomous service, because of their simple construction, are studied 
under general settings on dependent inter-arrival and inter-departure 
times, and their analysis is often based on methods of stochastic calcu- 
lus and the theory of martingales. The corresponding results for usual 
Markovian queueing systems or networks follow as a particular case 
of the corresponding results for queueing systems or networks with 
Poisson input and autonomous service. For different applications of 
queueing systems (networks) with autonomous service see e.g. [I], [3], 
[1] and 0. 

The assumption that the queueing mechanism is autonomous sub- 
stantially simplify the analysis. However, according to sample-path 
results of [I] this assumption can be removed. So, all of the main re- 
sults remain valid for quite general client /server networks without the 
special assumption that the service mechanism is autonomous. 

In the present paper we study client /server networks in a semi- 
Markov environment. There has been an increasing attention to queue- 
ing systems in a random environment in the literature (e.g. [9], [13] . 
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[T5] . [T9] . [26] . [29] . [31] and others). However, most of these papers 
mainly develop the theory and remain far from real-world applications. 

The aim of the present paper is twofold. First, we establish new 
theoretical results for client /server networks in semi-Markov environ- 
ment describing the behavior of queue-length processes of this network. 
Second, we show how these theoretical results can be applied to solve 
real-world problems. Some of these problems are solved in the present 
paper. Other ones will be solved in the future. 

The model of the network, which is considered in this paper, is very 
close to the models considered in [22J and pQ (see Figure 1). For a more 
general construction of network with two types of node and multiple 
customer classes see [3]. 

The description of the present model is based on that of the model 
of [lj. For other papers studying the models of client /server networks 
see also [10], [2j], [23], [21], [2J, [2J, [30], [35] and other papers. 

The previous assumptions of [I], that are also used in the present 
paper, are repeated below. 

The departure instants in the jth client station (j = 1,2, ... ,k) are 
denoted by £j t N,i, €j,N,i + €j,N,2, €j,N,i + £j,N,2 + €j,N,3y ■ ■ , and 

• each sequence {^yv.i, £j,N,2, ■ ■ •} forms a strictly stationary and 
ergodic sequence of random variables. 

The corresponding point process associated with departures from the 
client station j is denoted 



The total number of units in the network is N. The number N is 
a large parameter, and we assume that N increases indefinitely. This 
means that we assume that iV is a series parameter, and the series 
of models (with different N) are considered on the same probability 
space. 

It was assumed in [lj that the service time of each unit in the server 
station is an exponentially distributed random variable with a given pa- 
rameter A. In the present paper, the assumption is another. Under the 
assumption that an environment is random (Markov or semi-Markov), 
the parameter A is not longer a constant value. It is a random variable, 
taking values in dependence of environment states. (By environment 
states we mean the states of the corresponding Markov chain, which is 
described later.) The same is noted regarding the routing probability 
matrix. 
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Figure 1. An example of client /server network topology 
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There were the standard assumptions in [TJ that after a service com- 
pletion at the server station, a unit was transmitted to the client station 
j with probability pj, pj > 0, and Y^=iPj—^- These assumptions are 
not longer valid in the case of the system considered in this paper. The 
routing probabilities are assumed to be random, taking the values in 
dependence of the environment states as well. 

Let us first describe Markov states of semi-Markov environment, and 
then latter we describe the notion of semi-Markov environment as well. 
The network is assumed to be complete in the following sense. 

The server station and k client stations communicate by links. The 
number of links is k. Therefore, the Markov chain of the environment 
states is defined as follows. The number of possible environment states 
is assumed to be finite or countable infinite. These states are denoted 
by £ i, i — 1, 2, For the Markov or semi-Markov environment con- 
sidered later we shall also use the notation £(t). The meaning of this 
notation is the state of Markov chain in time t. The initial state is 
denoted by £q or £(0). 

The completeness of the network means that 

• for any j=l,2,. . . , k there exists ij > 1, such that Pj^y) > 0. 

This assumption is not used in our proofs explicitly. Nevertheless, it 
must be mentioned. If Pj{£^) = for all ij > 1, then the station j be- 
comes isolated and therefore not representative. In the above assump- 
tion by Pj(£i) we mean the value of probability Pj when the network 
is in state £j. It is assumed additionally that for any i = 1,2,..., the 
sum of probabilities Ylj=iPj{^i) = 1- The notation with similar mean- 
ing is used for parameter A. Namely, A(£j) is assumed to be strictly 
positive for any state however \j(£i) = X(£i)Pj(£i) can be equal to 
(because Pj(£i) need not be strictly positive in general). 

The results of [TJ are associated with the asymptotic analysis of a 
closed client /server network with a bottleneck station as N increases 
indefinitely. One of the main results of [lj was then developed in [3] for 
networks containing two types of node and multiple customer classes, 
where one of client stations was bottleneck. The results obtained in [TJ, 
[2] , [5] as well as in preceding paper [22] all are a theoretical contribu- 
tion to the theory of client /server computer networks with bottlenecks. 

The aim of the present paper is another. We follow towards perfor- 
mance analysis, and are aimed to use the known theoretical results of 
[22] and [TJ. However, the known theoretical results obtained in these 
papers are scanty for their immediate application to real telecommu- 
nication systems, whose parameters can change in time. Therefore, 
for the purpose of performance analysis, a substantial development of 



CLOSED QUEUEING NETWORKS IN SEMI-MARKOV ENVIRONMENT 7 



the earlier results related to this model is required. Therefore, before 
studying the behaviour of queueing networks in semi-Markov environ- 
ment, we first study the behaviour of this network in its special case of 
piece- wise deterministic environment. Then the results are extended 
to the more general situation of semi-Markov environment. 

The paper is organized as follows. In Section [21 the paper is mo- 
tivated by formulating the concrete real-world problems, where the 
results of the present paper can be applied. In section [31 necessary 
notions of piece-wise deterministic, Markov and semi-Markov environ- 
ments as well as classification of bottleneck stations are defined. In Sec- 
tion HI the semi-martingale decomposition for the queue-length process 
in semi-Markov environment is derived. Then in Section [51 the queue- 
length processes in piece-wise deterministic environment are studied. 
In fact we study "usual" closed client/server networks under general 
assumptions. In Section [6J, the results are extended to the case of semi- 
Markov environment. In Section [71 an example of an application of the 
main theorems of the present paper to client /server computer network 
with failing client stations is considered. In Section [HI new problems 
and monotonicity conditions associated with these problems for the 
case of Markov environment are discussed. In Section [HI the example 
of numerical study for the simplest network is considered. The results 
of the paper are concluded in Section [101 

2. Motivation 

In this section we motivate our study by real-world problems asso- 
ciated with performance analysis of the network, its separate stations 
or a subset of those stations. The main results of the present paper 
are the subject for many concrete practical problems having significant 
value. Two of them are considered in [6J, [7J. Article [6J assumes that 
client stations all are identical and subject to breakdowns. Article [7J 
discusses the similar problem for not identical client stations. As a 
client station is breakdown the parameters of the network are changed. 
One or other breakdown leads to bottleneck and risk of a destruction of 
the network performance. The aforementioned articles [5J, [7J establish 
confidence intervals helping to avoid with a given high probability that 
performance destruction. Some results of these articles are outlined in 
Section [7J of this paper. 

Another circle of problems is as follows. 

If a computer network operates for a given fixed time interval (0, 
T), then its performance characteristic (for example, payment for the 
increasing a given level of queue in a given client station, or in a subset 



8 



VYACHESLAV M. ABRAMOV 



of client stations) depends on the characteristics of this network at the 
initial time t — 0. Using the terminology of the present paper, these 
characteristics at the initial time t — specify the initial condition of 
the environment. For one initial condition, the measure of time that 
the queue-length is greater than a given fixed level L is x, and we must 
pay, say Cx. For other initial condition, this measure of time is y and 
the corresponding payment is Cy. 

On the other hand, the cost for initialization the first initial condition 
related to the above client station of the network is X, and that cost 
for initialization the second initial condition is Y. 

So, the total expenses in the first case are X + Cx, and in the second 
case Y + Cy. If X + Cx < Y + Cy, then we say that the first strategy 
is more profitable than the second one. In practical context, the first 
strategy can mean the first type repair of the network, or a failing client 
station, or a subset of failing client stations of the network, and the 
second strategy - the second type repair of that network, or a failing 
client station, or a subset of failing client stations correspondingly. For 
example, the first type repair can contain an additional prophylactical 
service and therefore to be more expensive that the second type repair 
(i.e. X >Y). 

3. Definition of semi-Markov environment and 
classification of bottleneck stations 

3.1. Semi-Markov environments. Let us now define a semi-Markov 
environment in Mathematical terms. For each state Si of the Markov 
chain let us define the sequence of independent and identically dis- 
tributed random variables: 

(3-1) Ci,l) Ci,2, ■ ■ ■ 

Definition 3.1. Let {Z n } be a Markov chain with finite or countable 
infinite states space {Si}. Let Z(t) be a random process, and let {o~ n } 
be an increasing sequence of random points, i.e. = a < U\ < . . .. 
Suppose that the process Z(t) is defined as follows: Z(a n ) = Z n , and 
Z(t) = Z{o n ) for all a n < t < <7 n +i- Suppose also that the increments 
o~ n +\ — o~ n , n=0,l,2,. . . , coincide in distribution with Q in , where the 
equality S(a n ) = Si defines the index i. Then the process Z(t) is called 
semi-Markov environment. 

The above semi-Markov environment Z(t) is assumed to be given on 
special probability space {VLz,Fz^z}, which in turn is contained in 
the common filtered probability space {Vt, T , F = (jF t ), P}. 
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The two significant special cases of semi-Markov environment are as 
follows. 

Definition 3.2. A semi-Markov environment is called piece- wise de- 
terministic, if 

• the sequences of (13.11) all are deterministic, i.e. Q tV = Zi, where zx 
is the same constant for all v. 

• the parameters of networks are not randomly dependent of the en- 
vironment states, i.e. Xj(S ) = Xj, Xj(£(ai)) = Xf\..., Xj(£(ai)) = 

Xj for all j=l,2,. . . , k, where X^\ xf , ...all are non-random 
constants. 

In other words, a piece-wise deterministic environment defines time 
dependent closed client /server network with fixed piece- wise constant 
parameters depending on time. 

Definition 3.3. In the case where the sequences (13. ip consist of in- 
dependent and exponentially distributed random variables with rates 
Zi, 2=1,2,. . . , then the semi-Markov environment Z(t) is called Markov 
environment. 

Another equivalent definition of Markov environment and indepen- 
dent of Definition 13. II is as follows. Let Z(t) be a homogeneous Markov 
process with initial state £(0) and transition probabilities z^ m At + 
o(At) from the state £ to the state £ m (I ^ m) during a small time 
interval (t, t + At), and there is probability 1 — Ylim^i z i,m^t + o(At) 
to stay in the same state Si during the same time interval (t, t + At). 

In many papers on queueing theory, telecommunication systems, in- 
ference of stochastic processes, statistics and other areas, the above 
Markov environment is often associated with Markov Modulated Pois- 
son Process. The phrases Markov environment, piece-wise determinis- 
tic environment and semi-Markov environment are more appropriate 
in the context of the present paper. 

3.2. Classification of bottleneck stations. There was mentioned 
that S(t) denotes the state of the semi-Markov environment in time 
t. For example, the equality £(4) = £ 5 means that in time t — 4 
the associated Markov chain is in state £5. We also use the following 
notation: A j (£(t)) = X(S(t))pj(£(t)). 

In the sequel we shall also use the notation X(t), Pj{t) and corre- 
spondingly, Xj(t). It is worth noting, that the definition of X(t) and 
A (£(£)) and consequently Xj(t) and A j (£(£)) have the different mean- 
ing. In general A(£(t)) ^ A(t) and Xj(£(t)) ^ Xj(t). Xj(t) and Xj{£{t)) 
both are random parameters, and Xj(t) = Xj(t,u>) where u G Q, while 
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Xj(£(t)) = Xj(t,u>z) where ujz G Qz- (In the sequel this dependence 
upon uj z will be always implied, but shown explicitly only in the cases 
where it is necessary.) 

More specific explanation of the above difference is as follows. If 
at the initial time moment t — all of units are in the server sta- 
tion, then the input rate to the jth client station is Xj(0)N. (Xj(Q) 
is the individual rate of each unit arriving to the client station j, and 
therefore the rate between arrivals is Xj(0)N.) The time parameter 
in parentheses is associated with the state of Markov environment in 
time 0. Specifically, if at the initial time moment t — all of units 
are in the server station, then Xj(0) = Xj(£(0)). If the network is con- 
sidered without Markov or semi-Markov environment, then that initial 
arrival rate is XjN which is associated with the individual rate A-,- of 
each unit of the server station. If at the initial time moment there are 
aN units in the server station, a < 1, then for arrival process to any 
client station j of a standard network (without random environment) 
we also use the notation Aj(0). However, the meaning of Aj(0) is not 
longer the individual rate of each unit at time t = arriving to the 
station j. More specifically, Xj(0) = XjCe, where Xj is the individual 
rate of each unit at time t = arriving to the station j. The meaning 
of Xj(t) is similar. The only difference that it is said about an arbitrary 
time t. For example, if there are a(t)N units in the server station in 
time t, a(t) < 1, then Xj(t) = Xja(t). Resuming the above, Xj(£(t)) 
is associated with individual service rate of each unit, while Xj(t) is a 
recalculated (relative) rate depending on the state of the queue-length 
processes in time t, as explained above. 

Regarding the departure (service) rates our assumption in the paper 
is as follows. The departure rate of the jth client station is assumed to 
be independent of semi-Markov environment as well as of input rates 
Xj(t). Specifically, 

• it is assumed that the the expectation of service (inter-departure) 
time in the jth client station is ^Cj,N,i = for all 1=1,2,. . . 

Therefore, if at the initial time moment t = all of units are in 
the server station, then the load parameter of the jth client station is 
Pj(0) = and in the case where there is no semi-Markov environ- 

ment, pj(0) = Pj = %L. 

In the case of network in Markov environment, the meaning of the 
notation Xj(t) is the same as well. If there are Na(t) units in the server 
station in time t, a(t) < 1, and the rate of arrival of a unit from the 
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server to the client station j is Aj (£(£)), then Xj(t) = Xj(S(t))a(t). 
Then the load of the jth client station in time t is pj(t) = 

Now introduce necessary definitions. The first two definitions are 
related to both standard client /server networks and client/server net- 
works in semi-Markov environment. 

Definition 3.4. The client station j is called locally non-bottleneck in 
time t if Pj(t) < 1. Otherwise, the jth. client station is called locally 
bottleneck in time t. A client station locally (non-) bottleneck in time 
will be also called initially (non-) bottleneck. 

Definition 3.5. The client station j is called non-bottleneck in time 
interval [t\, £2] if it is locally non-bottleneck in all points of this interval. 
Otherwise, if there is a point t* G [£1, £2] such that pj(t*) > 1, then the 
client station is called bottleneck in time interval [£1, £2]. A client station 
is called (non-) bottleneck if it is (non-) bottleneck for all t. 

The special definition for standard client/server networks (without 
semi-Markov environment) is as follows. 

Definition 3.6. A client station is called absolutely non-bottleneck if 
it is a locally non-bottleneck station at the moment when all of units 
are in the server station. Otherwise, a client station is called absolutely 
bottleneck. 

Clearly, that absolutely non-bottleneck client station j is a non- 
bottleneck client station, because then for all t we have Xj(t) < pj. 
In the next section we prove that absolutely bottleneck client station 
is a bottleneck client station as well. That is, if a client station is 
currently locally bottleneck station, then it never can become a locally 
non-bottleneck. This means that the client station is forever bottle- 
neck. 

Definition 13.61 can be extended to network stations in semi-Markov 
environment for an arbitrary time t. Specifically, we have the following 
definition. 

Definition 3.7. The client station j of a network in random environ- 
ment is called absolutely non-bottleneck in time t if Aj (£(£)) < fij. In 
other words, the client station j is absolutely non-bottleneck in time t 
if in that time t the network belongs to some state Si of the environ- 
ment, i.e. £{t) = Si, and in this state \j(£i) < fij. Otherwise, this 
client station is called absolutely bottleneck in time t. 

The last notion enables us to judge on the behavior of client stations 
in random intervals [crj, cr i+1 ), where the network is in given state Si. 
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Recall that o* is a time instant when the state of semi-Markov environ- 
ment is changed. So, during the random interval [o*, 0*4.1) the network 
is in a fixed state of the semi-Markov environment. 

4. Queue-length processes in the client stations of 
networks with semi-markov environment 

Consider a client station j (j = 1,2, ...,k). Let Qj^if) denote a 
queue-length there in time t. Assume that at the initial time instant 
t = 0, all of units are in the server station, i.e. <3j,jv(0) = for all 
j = 1,2, ... ,k. This is the simplest case, and we start from its study. 
For t > 0, 

(4.1) Q j>N (t) = A j>N (t) -D j>N (t), 

where Aj^if) is the arrival process to client station j, and Dj^it) 
is the departure process from that client station j. The equation for 
departure process is as follows. Let 

00 ( i ^ 

SjAt) = J2 1 ) J2^ N > i ^ 1 \ ' j = 1,2, . . . 

8=1 [ l=\ ) 

be a point process associated with consecutive departures from the jth 
client station. Then, 



Jo 



n(s) 



(4.2) 



= S j>N (t) ~ / HQiAs-) = 0}dSjAs), 
Jo 

j = l,2,...,k. 

The definition of the departure process given by (14.21) is as in pp. How- 
ever, the construction of arrival process is more difficult. 
Specifically, 

(4.3) AjAt) = J o ^llN-^Q l As-)>i\dn jti (s,u z ). 

The processes {iTj^s, d>z)}, i = 1,2, . . . , N, appearing in relation (14.31) 
are a collection of conditionally independent Poisson processes with 
parameters depending on uz- This means the following. Assume that 
for a given realization loz we have a sequence: 0< 01 < a 2 <■ . . , 
and for some n, a n < s < a n +i. Assume that £ (o n ) = £[. Then 
{7Tj t i(s, ojz)}, i = 1, 2, . . . , N, is the sequence of Poisson processes with 
the parameter \j(£i). That is the rate of Poisson process depends on 
the state of semi-Markov environment in time s. This is just the main 
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difference between the consideration of p], where Xj, j = 1,2, ... ,k, 
were non-random constants. 

Relation H4.3[) can be then rewritten 

N 



rtA<T v I | 

(4.4) A hN {t) = J2 M N ~ J2 QiAs-) > * \ d7T jAv (s) 



where ttj^^s) is an associated sequence of (conditionally) indepen- 
dent Poisson process with parameter depending on the state of the 
semi-Markov environment £(cr v -i). (Here in (14.41) and later we use the 
standard notation for a minimum of two numbers: a Ab = min(a, b).) 

Relations (14.11) and (14. 2p enable us to write the following represen- 
tation for the queue-length process Qj^(t): 

(4.5) Q jtN (t) = A hN (t) - S hN (t) + / I{Q hN ( s -) = 0}dS jtN (s). 



This implies that Qj^it) is the normal reflection of the process 

(4.6) X jtN (t) = A jtN (t)-S j>N (t), X jtN (0) = Q 

at zero. More accurately, Qj,jv(t) is a nonnegative solution of the Sko- 
rokhod problem (see [33] as well as [8], [32], [34 J and others) on the 
normal reflection of the process Xj jN (t) at zero (for the detailed argu- 
ments see [22] )• Recall that according to the Skorokhod problem, 

(4.7) f l{Q jjN (s-) = 0}dS hN (s) = -MX jiN ( 8 ), 
Jo s ^ 

and Qj,N(t) has the representation 

(4.8) Q jtN {t) = X hN {t) - MX hN {s). 

s<t 

In the sequel, it is convenient to use the notation: 

s<t 

for any cadlag function X(t) satisfying X(0) = (see e.g. [22])- Ac- 
cording to this notation, (14.81) can be rewritten Qj^if) = ^tiXj^). 

Next, we take into account that the process Aj^(t) is a semimartin- 
gale adapted with respect to the filtration Tt- Let Aj^if) denote the 
compensator of Aj^it) and MA jN (t) denote the square integrable mar- 
tingale of Aj t N(t) in the Doob-Meyer semimartingale decomposition: 
Aj,N(t) = Aj^{t) + Mi. N (t), j = 1, 2, . . . , k. Then, the process Xj^t) 
given by (14.61) can be represented 

(4.9) X j>N (t) = A hN (t) - S hN (t) + M A . N (t), 



14 VYACHESLAV M. ABRAMOV 

where 

(4.10) A j;N {t) = J2 / A i(^-0) \n -Y,QiA s )\ ds - 

1=1 J<Jl-\M \ i= i J 

The details for last formula (14.101) can be obtained from [T4] or [27], 
Theorem 1.6.1. 

Let us now study asymptotic properties of the normalized queue- 
lengths qj,N{t) = ® 3 '^^ as N — > 00. For normalized processes we will 

use small Latin letters. For example, Xj^it) = Xl '^ , aj^t) = Aj ^^ ; 
and so on. 

Then, from (14.91) we have 

(4.11) x jiN (t) = a j>N (t) - s jiN (t) + m AjN (t), j = 1, 2, . . . , k. 

Let us derive a relation for P- limjv^oo Xj^(t). (P- lim denotes the limit 
in probability.) 

From Lenglart-Rebolledo inequality (e.g. Liptser and Shiryayev [27]). 
we have: 
(4.12) 

P { sup \m AjN (t)\ > 5} = P { sup \A^ N (s) - A^ N (s)\ > Sn\ 
lO<s<t ) Lo<s<< J 

<^ + F{A hN (t)>eN 2 }, 

where e = t(N) vanishes such that eN — > 00. Then, by virtue of (I4.10p . 
the term F{A jtN (t) > eN 2 } = ¥{a jtN (t) > eN} vanishes as well, and 
for any small 5 > the fraction 4s vanishes. Therefore, 

(4.13) P" lim m AiN {t) = 

for all j — 1, 2, . . . , k and t > 0. 

Next, according to the assumption above, P-limjv-»oo s j,N(t) = [ijt. 
Therefore, 

Xj(t) = P" lim Xj ^(t) 

= P" lim a,j tv(£) — fijt 

(4.14) A ~ N 



/ 


A;(£(^-i)) j 







i=i 



ds. 
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Representation (14.141) is the extension of the similar result of pQ for 
queue-length processes in client stations of standard client /server net- 
works. Recall that the representation obtained in [Tj and [22] is 









(4.15) 


x s {t) = [ 






Jo 





k 
i=l 



ds. 



As we can see, the representations given by (14. 14[) and (I4.15P are 
similar. The only difference is in the presence of infinite sum containing 
the integrals with random upper and lower bounds, and (14.151) is a 
particular case of (I4.14p . 



5. Bottleneck analysis in the case of a piece-wise 
constant environment 

In this section we discuss the behavior of the queue-length processes 
in a large closed client /server network in the case of piece- wise constant 
environment. For this case relation (I4.14p reduces to 



(5.1) Xj (t) = 



Xi > — 



i=l 



ds 



+ 





H 


/ <7 r 





1=1 



ds, 



where r is the number of the state changes before time t. 

Therefore, the bottleneck analysis of the network in piece-wise con- 
stant environment reduces to the analysis of the network in traditional 
formulation (without random environment) in some given time inter- 
vals such as [<7j_i,0i), /=1,2,. . . ,r, or [a r ,t). 

Notice, that the bottleneck analysis of the Markovian client /server 
model has been originally studied by Kogan and Liptser [22]. Then 
these results were extended for the case of autonomous service mech- 
anism in client stations in [1]. However, the results obtained in both 
of these papers are related to a single special case and are not enough 
for the purpose of our performance analysis. Therefore we will study 
all possible cases including the behaviour of the network under differ- 
ent initial lengths of queues in client stations and several absolutely 
bottleneck and absolutely non-bottleneck client stations. 

In [22] and [1] there has only been considered the case where in the 
initial time moment t = all of units are in the server station (i.e. 
the client stations all are empty) and only one (the kth) client station 
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is a bottleneck station. (In this particular case the notions of (non- 
bottleneck and absolutely (non-)bottleneck client station coincide.) 
Specifically, there has been proved the following result in pQ. 

Proposition 5.1. Let S* N (t) = inf{s > : Sj^ N (s) = Sj^ N (t)}, and 
Qj,N{t) denotes the queue-length in the jth client station in time t. 
Under the assumption that the kth client section is bottleneck, for j = 
1, 2, . . . , k — 1 and for any t > we have: 



(5.2) lim nQiAS*, N (t)} = 0} = 1 - Pj (t), 

lim / Pj {s)F{Q jN {s) = l}ds 

lim I V{Q jtN [S* N (s)] = l + l}ds, 
vr -*°° Jo 



(5.3) 



where 



N- 

/ = 0,1,... 



(5.4) Pj (t) = Pj (0)[l -q(t)}, 

(5.5) q(t) = ( 1 \-] (1 - e- pk ^ kt ). 



Pk(0) 

In the case of Markovian network, relations (15. 2p . (15. 3p can be re- 
solved explicitly, and queue-length distribution in the non-bottleneck 
stations is the time-dependent geometric distribution PQ, [2"2"] . 

The meaning of 1 — q(t) in (15.41) is the limiting fraction of units 
remaining at the server station in time t as iV — ► oo. For example, in 
the case where pfc(0) = 1 this fraction remains the same at any time t 
as initially, that is, as iV large, the number of units in the server station 
remains asymptotically equivalent to N. However, if pfc(0) > 1, then 
the number of units in the server station in time t is asymptotically 
equivalent to N[l — q(t)]. Then, the number of units remaining in 
bottleneck station in time t is asymptotically equivalent to Nq(t). 

Let us study various cases of the client /server network with bottle- 
necks. These cases will be studied in order of increasing complexity. 
We possibly shorten the proofs. 

Assuming that the initial condition of the network is the same as 
in Proposition 15.11 (i.e. at the initial time t = all of units are at the 
server station), let us study the case where the client stations 1,2,. . . ,ko 
are non-bottleneck, while the rest client stations ko + 1, . . . , k are bot- 
tleneck. In this case we have the following result. 



CLOSED QUEUEING NETWORKS IN SEMI-MARKOV ENVIRONMENT 17 



Proposition 5.2. Under the assumption that the client sections ko + 
1, . . . , k are bottleneck, for j = 1, 2, . . . , k and for any t > we have: 

(5.6) lim nQjASjAt)} = 0} = 1 - Pj (t), 



(5.7) 



lim / Pj (s)¥{Q jjN (s) = l}ds 

lim / F{Q j>N [S* N (s)) = l + l}ds, 



N 

1 = 0,1,... 



where 

(5.8) Pj (t) = Pj (0)[l -q(t)], 



(5.9) q(t) 




1 " I I 1 - exp 



-t J2 k 

v=ko+l 



Note, that equations (I5.6P and (15.71) are the same as (15.21) and (15. 3p . 
and the main difference between Proposition 15.11 and Proposition 15.21 
is only in the expression for q(t). The difference between expressions 
(15.41) . (I5.5p and (15.81) . (15.91) can be easily explained in the framework of 
the proof of Proposition 15.21 

Considering all of the bottleneck client stations as a separate subsys- 
tem, one can notice that the arrival rate to this subsystem is Y^t=k +i 
and the service rate (the sum of reciprocals of the expected inter- 
departure times) is Ylt=k +i A*«- This subsystem can be thought as 
a bottleneck station with the load 

Ylv=k +1 ^ v 



Ek 
v=k +l ^ % 



To be specific, note the method of Sections 2, 3, and 4 of [T] leads to 
the same equations, and the equations for normalized queue-lengths 
in the bottleneck stations all are an elementary extension of the case 
considered in [T]. 

Therefore Proposition 15.21 is an elementary extension of Proposition 
15.11 and its proof is the same as in [1] or [22] . Notice, that the number 
of units remaining at the bottleneck station v, v = k + 1, . . . , k in time 
t is asymptotically equal to 

(5.10) Nq v {t) = N ([\ v - Pv ]t - \ v q{s)ds 
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Recall the main elements of the known proof for the representation 
q(t) given by (15.51) . and consequently explain the proof of (15.101) . We 
use the notation similar to that of the earlier papers |T] and [22J. 

The difference between arrival and departure processes in the jth 
client station is denoted Xj jN (t) = Aj N {t) — Sj^if) and its normal- 

X ■ (t) 

ization Xj^(t) = 3 ' N y . Let Xj(t), j = 1,2, ...,k, denote the limit 
in probability of Xj^(t), as iV — > oo. The queue-length in the jth 
client station in time t is denoted Qj y N{t), its normalization is denoted 
Qj,N(t) = , and the limit of qj,N(t) in probability as N — > oo, is 

denoted qj{t). 

Next, let = X{t) — m.i s < t X(s) for any cadlag function X 

satisfying the condition X(0) = 0. The functional <3?t(X) has been in- 
troduced in |22j . It characterizes a solution of the Skorokhod problem 
on normal reflection at zero. Therefore $> t (xj), being the functional 
$ applied to the function Xj(t), describes the dynamic of normalized 
queue-length in the jth client station under the "usual" initial condi- 
tions given in Propositions 15 . II and 15.21 Under these "usual" conditions, 
the functions Xj(t), j = 1,2, ... ,k satisfy the system of equations: 



(5.11) Xj{t) 




-fij>d8, j = l,2, 



Note, that the normalized functions Xj(t), j = 1,2, ...,k are usual 
(non-random) continuous functions, and (15. lip characterizes a usual 
system of linear differential equations. 

The statement of Proposition 15.11 is based on the solution of the 
system of these equations. (It is proved in [22] that there is a unique 
solution of the system of equations (15.111) .) More specifically, in the 
case where the node k is bottleneck, mi s < t Xk(s) = Xk(0) = 0, and we 
therefore have $ s (xfc) = x^(s). In other words, $ s (xfc) can be replaced 
by x k (s) in these equations of (15. lip . 

The solution of the system of equations (15. lip is Xj(t)=0 for the 
non-bottleneck stations j = 1, 2, . . . , k — 1, and Xk(t) = q(t) for the 
bottleneck station j = k, where q(t) is given by (15.51) . In the case 
of several bottleneck stations, we write the similar equation for the 
dynamic of the normalized cumulated queue-length process in all of 
the bottleneck stations. Specifically, 




k 

i=k +i 



v=k +l 
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gives solution (15.91) for q(t) in the similar statement of Proposition 15.21 
Then, the solution of system (15. lip for the bottleneck client stations 
v=k + 1,/cq + 2. . . , k is given by 



where q(t) is defined by (15.91) . and the queue- lengths in the bottleneck 
stations are asymptotically evaluated by relation (I5.10p . 

Now, we discuss the behavior of the network, in which the client 
stations are not initially empty. This is the next step of the extension 
of the original result of Proposition 15.11 The analysis of cases related 
to initially not empty queues is much more difficult. Therefore, we 
start from the simplest case of the network containing only one client 
station, i.e. k = 1. 

Let ft < 1 be a positive real number, and let us assume that the 
initial number of units in this client station is asymptotically equivalent 
to iVft. Consider the following two cases: (i) the client station is 
initially bottleneck, i.e. Ai(0) > H\, and (ii) the client station is initially 
non-bottleneck, i.e. Ai(0) < 

Case (i) is relatively simple. It is a simple extension of the cases con- 
sidered above. Specifically, we have the following system of equations: 



The normalized queue-length qi(t) in this client station is qi{t) = Xi(t). 

Case (i) can be easily extended to a more general case of k initially 
bottleneck client stations. Let ft, j = 1,2,..., k, denote nonnegative 
real numbers, and ft + ft + . . . + ft < 1. Assume then that the initial 
number of units in the jth client station is asymptotically equivalent 
to Nf3j. Let q(t) denote the cumulated normalized queue- length in all 
of client stations. 

Proposition 5.3. Assume that all client stations are initially bottle- 
neck, and the initial queue-lengths in client stations are asymptotically 
equivalent to iVft, iVft,. ■ ■ , N(3 k correspondingly (ft + f3 2 + • • • + flk < 




(5.12) 



n(t) = A + (i-AWt), 

*i(*)= / {Ai(0)[l-zi(s)]-//i}ds 



Jo 



From (I5.12p we have the following solution: 



(5.13) Xl (t) = ft + (1 - ft) (^oT ) (1 " e ~ Xli0)t) 
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1), as N — > oo. Then, 

k / k \ 

C5 14) ' ?=1 5=1 

r(tH| eLa,(o) Ul - exp 



3=1 

and the normalized queue-length in the jth client station is defined as 



(5.15) qj(t) = ft + {l ~ Z>j - ^ ~ A ^°) I r ^ ds ) ■ 

Let us now discuss case (ii). This case is also described by system of 
equations (15.12p . and the dynamic of the process x±(t) is therefore sim- 
ilar to the case considered above. However, this case is more delicate. 
The client station is initially non-bottleneck, i.e. Ai(0) < Hu and the 
function Xi(t) is therefore decreasing in the right side of 0. According 
to the convention, the initial value of queue is asymptotically equiva- 
lent to PiN, and therefore Ai(0) = (1 — /#l)A*. Then the meaning of A* 
is a maximally possible rate of units arriving from the server station to 
client station, when all of units are in the server station and the client 
station is empty. Then the client station is absolutely non-bottleneck 
if X{ < Hi, and it is absolutely bottleneck if A* > fii- 

Consider first the case of an absolutely bottleneck station, i.e. A^ > 
Hi. In this case, from the solution given by (15.131) we have 

Ai(0)-//i 



lim q 1 (t) = P 1 + (l-P 



(5.16) 



Ai(0) 
Ai(0) - Hii 1 - Pi) 

Ai(0) 
X{ - Hi 



The meaning of the last result is the following. Let Nj3\ be an asymp- 
totic value of the queue-length in the client station, when this queue 
station (at the first time) becomes a locally bottleneck station. Then, 
according to (15.161) the normalized queue-length in the client station 
approaches to this level 8\ = A \, Ml as t — ► oo. Notice, that the same 
level for normalized queue-length is achieved for an initially bottleneck 
station in case (i). The result of (15.161) is also supported by results 
(15.41) and (15.51) of Proposition 15.11 Thus, for a bottleneck station, 
the same level is asymptotically achieved independently of an initial 
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queue-length. For this reason for any absolutely bottleneck station in 
which XI > fii we do not distinguish between two cases Aj(0) < fa 
and \j(0) > fa, and absolutely bottleneck client station is always a 
bottleneck station. 

Consider now the case of an absolutely non-bottleneck client station 
(and therefore non-bottleneck client station) where A^ < fi\. Then, 
according to the same calculation as in (15.161) we have 

(5.17) limx 1 W = ^<0. 

Therefore, there exists the time instant T\ when the normalized queue- 
length becomes at the first time empty. For this time instant we have 
the equation 

(5 . 18 , n= * log( 1+ "' A '<°) 



\{0) a \ (1 - ft)(A!(0) - ft) 

Let us extend the result of case (ii) for a network with k initially 
non-bottleneck client stations, all satisfying the condition Xj(0) < fa;, 
j = 1, 2, . . . , k. It is assumed that the initial number of units in the 
jth client station is asymptotically equivalent to Nj3j (p% + /3 2 +■ ■ ■ + 
Pk < 1) as N — > oo. Assume also that the first k client stations are 
(absolutely) non-bottleneck, i.e. A* = Aj(0)(l— /?i —fy — ... — (3k) < fa, 
j = 1, 2, . . . , ko, while the rest k — k client stations are (absolutely) 
bottleneck, i.e. A* = \ v (0)(l-(3i —p2 — --- — Pk) > fa, v = k + 1, k + 
2, — , k. 

Similarly to (15.121) we have the following system of equations 

k / k \ 

j=i \ j=i j 



(5.19) 

z(t) 



t ( k 1 

I |(i-^))E A i(°)-E^j ds ' 



and for z(t) we have the solution 




k 



(5.20) = ( ^ =i( , Aj(0) " ' 111- rxp 

Then for Xj(t), j = 1,2, ... , k, we have the following solutions: 
(5.21) 

Xj (t) = p j +[i-±2P j \[ (\M - fa)t - Ai(o) J* z ( s ) ds ) 
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(Recall that Xj (t) = P- lim^oo A ^- s ^\ j = 1,2,..., fc.) 

However, since the first ko client stations are non-bottleneck, then 
the equality qj(t) = Xj(t) for the normalized queue- lengths in client 
stations is valid only for the values t of the interval < t < Ti, where 
the value T\ can be found from (I5.2ip as 

(5.22) n = min inf{t : Xj(t) < 0}. 

l<j'<fco 

Let jo = arg minx<j<fe inf{£ : Xj(t) < 0}. Then the normalized queue- 
length process qj Q (t) is as follows. For < t < Ti, qj (t) = Xj (t), and 
for t > T\ it satisfies the equation 



(5.23) x j0 (t) 



o 



ds, 



where § s (xj ) = x jo( s ) ~ mr o<«< s x jo (u). 

Together with (15.231) for all remaining j =1,2,. . . , j — 1, j + 1,. . . 
k and t > T\ we have: 



(5.24) 

z(t) 



T Tl ((l-^))^A,(r 1 )-^^}d S , 



and similarly to fl 5 . 2 1) for z(t) we have the solution 
(5.25) 



-(t-rO^A^n) 
Therefore for j =1,2,. . . , j -l, jo+1,. . . , k and £ > T\ we obtain 



(5.26) 



(t) =x J ( n )+ I l-^x^n)) 



x ( (Ai(ri) - /ij)(t - n) - Xjin) / 2(s)ds 



o 



and since Xj (t) is nonnegative, for j = jo and t > T\ we obtain 
(5.27) **(*) = °. 

Thus the dimension of the system is decreased by 1, and the pro- 
cedure can be repeated similarly. Specifically, again since the client 
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stations j =1,2,. . . , jo — 1, jo + 1,. . . , k all are non-bottleneck sta- 
tions, then the equality qj(t) = Xj(t) for the normalized queue- length 
processes is valid for all t < r 2 , where r 2 is defined as 

(5.28) r 2 = min inf{£ : xj(t) < 0}. 

l<jWo<fca 

Setting now jx = axg minx<jyj <jfc inf {< : Xj(t) < 0} one can continue 
this procedure to find r 3 ,. . . , Tfc and then to know the behaviour of the 
queue-length processes in all of (non-bottleneck and absolutely bottle- 
neck) client stations for all t. 

The considered extension of case (ii) is in fact the general case in 
which amongst k client stations there are ko non-bottleneck client sta- 
tions, and the rest k — ko stations are absolutely bottleneck (i.e. part 
of them can be initially non-bottleneck), and all k client stations are 
with arbitrarily large initial queue-lengths. 

The result can be formulated as follows. 

Theorem 5.4. Assume that there are k client stations, where ko client 
stations (non necessarily the first ones) are absolutely non-bottleneck, 
and the initial normalized queue-length in all of these k client stations 
are (3j in limit as N — > oo correspondingly (j = 1,2, ... ,k). Then 
there are time instants r% < r 2 < . . . < Tfc when the normalized queue- 
lengths in these k absolutely non-bottleneck stations correspondingly 
achieves zero and remains then to stay at zero. These points as well as 
the normalized queue-lengths in time t in the rest k — k stations are 
defined according to Scheme 15.51 below. 

Scheme 5.5. Consider the system 
(5.29) 



Xj 



(t) = Xj {0) + ^1 - X>(0)j ((A,(0) - N )t - A,(0) jT z(s)ds^J , 



j = l,2,...,k, 
where Xj(0) = 0j, and 



E? = i(A 3 -(o)-^-) \ / 



(5.30) z(t) = 1 - exp 



Then, 



E =1 A,(0) 



n = min inf{t : Xj(t) < 0}, 
i<jf<fc 



k 



and the argument t in (I5.29j) and (15.301) belongs to the interval [0,Ti] . In 
this case the normalized queue-length at the jth client station, qj (t) = 
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Xj(t) for all j=l,2,. . . ,k. Let jo = argmini<j<fc inf {t : Xj(t) < 0}. Then 
qjo(t)=0 for all t > r x . 

In the next step, we consider the system of k — 1 equations, and 
£ > Tx (the equation for Xj (t), t > Ti, is excluded): 



(5.31) 



=^0~i) + f 1 - ^ ^( r i) J 



x ( Oj( T i) - - T i) - A i( r y ^( s ) ds 



t-n 



where 

(5.32) *(t) 
Then, 



T2 = min inf{£ > Ti : £,(£) < 0}, 
i<j<fc J 

ji = arg min inf{t > T\ : Xj(t) < 0}, 
i<j<fc 



and the argument t in (15.311) and (I5.32p belongs to the interval [r 1 ,r 2 ], 
and subscript j in these equations is j = 1, 2, . . . , j — 1, j + 1, . . . , k. 
In this case qj(t) = Xj(t), and q^t) = for all t > r 2 . 

Consequently excluding equations one-by-one, then the Zth instant 
^7, 2 < I < fco, is defined from the system 
(5.33) 



x 



where 



(5.34) 



/t—n-i 
z(s)ds ) , 



z(t) = 1 ■ {Xji ' ;i J /,f) | x 

^j^jo,jl,-,jl-2 X J {li l ? 

x I 1 - exp -t Y A i( r «-i) 
j¥=h,h,—,3i-2 
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Then, 



n 



3i-i 



min inf{t > T;_i : xAt) < 0}, 

l<j<k 

arg min inf{£ > tj_i : Xj(t) < 0}, 

j^JO ,jl,—,jl-2 



and the argument t in (I5.33P and (15.341) belongs to the interval [rj_i,rj], 
and subscript j in these equations takes the values from 1 to k but jo, 
jir ■ ■ , Jz— 2- in this case the normalized queue- lengths qj(t) = Xj(t), 
and Qji_ 1 (t)=0 for all t > 77. 



Theorem 15.41 containing Scheme 15.51 is easily applied to networks 
with piece-wise deterministic environment. Specifically, the theorem is 
applied to each of the intervals [erj_i, ai), I — 1,2, ... ,r and [a r , t). 

6. The main theorem on queue-length processes in the 
client stations of networks with semi- markov 
environment 



The statement of Theorem 15.41 is easily adapted to client /server net- 
works in semi-Markov environment. The difference between the ap- 
proach to the piece-wise deterministic environment and semi-Markov 
environment is only that the points 0\ are random, and general relation 
(I4.14p rather than (15.11) must be used. 

For example, considering the first term of (I4.14p . we have 

Xj{p\ A t) 

"<7tA(<TlAt) 

' A^fa-i)) <1 



(6.1) 



E 

1=1 



<T!-iA(<JiAt) 




ds 





W (0)) | 


Jo 





k 
i=l 



ds, 



and the extension the above theory of Section [S] is elementary. For 
example one can make the assumption about arbitrary initial condi- 
tions in client stations and arbitrary number of bottleneck stations. 
The same theory of Section [5] can be developed for any interval with 
random lower and upper bounds as well, such that 

k 



rat At 


A;(£(^_i)) | 


'cri- 1 At 





1 



5> 




Hi 



ds, 



resulting in adaptation of the theory of Section [5] to client /server net- 
works in semi-Markov environment. 



26 



VYACHESLAV M. ABRAMOV 



The theorem below is the adaptation of Theorem 15.41 to the case of 
a random interval [0, o\ A t). (The extension of Theorem 15.41 to the 
other intervals [o"i_i A t, o\ A t), 1=1,2,. . . , is similar.) 



Theorem 6.1. Assume that amongst k client stations, there are k 
absolutely non-bottleneck in time 0, and the rest k — /c client stations 
are absolutely bottleneck in time 0. Assume that the initial normalized 
queue-lengths in all of these k client stations are f3j in limit as N — ► oo 
correspondingly (j = 1,2, ... ,k). Then there are time instants T\ < 
t~2 < • • • < Tfc which are defined recurrently by Scheme \5. 51 of Theorem 
5.4 We only take into account the values 7$ satisfying the inequality 
Ti < 0i A t, i.e. we set I = t{o~\) = max{z : n < 01 A t}. Then, the 
only instants T\ < r 2 < . . . < are taken into account in this theorem. 
The main relation (15.291) of Scheme 15.51 now looks 



Xj (a 1 A t) =Xj(0) + ^1 - ^Xj(O)^ 



(6.2) / rcr 1 M 

x I (Aj(0) - {ij)(<Ti At) - Aj-(0) / z(s)ds ) , 
3 = 1,2,. ..,k, 



where Xj(0) = (3j, and 



(6.3) z(t) 



ELa,(o) 




cxp 



3=1 



The other relations of this theorem are defined similarly to the corre- 
sponding relations of Scheme 15.51 where only argument t is replaced by 
01 At in the corresponding places. Specifically, (I5.3ip now looks 



(01 At) = xjin) + 1 1 - ^2 x i( r i) I 

V j¥=30 / 



x 



/(o-iAt)--n 
z(s)ds 
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where z(t) is defined by ( 15.321) . and (I5.33f) now looks 
(6.4) 

Xj ( ai At) — Xjin-t) + I 1 - x i( r '-i) x 

(\j{n-i) ~ At) - n-i) ~ Ay(r z _i) y 2:(s)g?s 

where z(t) is defined by ( 15. 34ft . and Z = 1, 2, . . . , £. 

7. Application: Performance analysis of symmetric large 
closed client/server computer networks with 
unreliable client stations 

In this section we discuss application of the above main theorem for 
symmetric large closed client server computer networks. 

7.1. Formulation of the problem. We consider a network with k 
identical servers. We assume that all of the processes started at zero, 
i.e. Aj t ]y(0) = Sj t isr(0) = Q?',iv(0) = 0, and the following other as- 
sumptions and notation are used. The service time of each unit in the 
server station is exponentially distributed with parameter A. There 
are k client stations in total, and each of client stations is a subject to 
breakdown. A lifetime of each client station has the probability dis- 
tribution G(x). In this case the moments of breakdown are associated 
with change states of semi-Markov environment, and this example is 
one of applications of the above theory. 
The process 5j,jv(t) satisfies the condition 

(7.1) p(lim \ = 1> 

(The equivalent form of condition (17. ip is considered in Section [3j It 
is assumed here that fix = fx 2 — ■ ■ ■ = A*fe = H-) 

The relations between parameters A, /i and k are assumed to be 

< 7 - 2 > h < h 

and 

(7.3) - > 1. 

Condition (17. 2p means that all of the client stations are initially non- 
bottleneck. Condition (17. 3p means that after one of other breakdown 
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all of the client stations become bottleneck. Denote 

lo = max { I : — > 1 1 
I ¥ J 

the maximum number of available client stations when the client sta- 
tions all are bottleneck. Then for all / < lo the rest / client stations 
will be bottleneck as well. 

Let a < 1 be a given positive number. We say that the network is at 
risk if the total number of units in client stations exceeds the value aN. 
Assuming that at the initial time moment all of the units are in the 
server stations, the aim of this section is to find a confidence interval [0, 
9) such that with a given high probability P (say P = 0.95) the network 
will not be at risk during that time interval [0, 9). The parameter N 
is assumed to be large. Therefore we study asymptotic solution of this 
problem as N — > oo, that is we study limiting (as iV — > oo) normalized 
cumulative queue-length process in client stations. This problem is 
hard in general. Here we consider a particular case of lo — 1. 



7.2. Solution of the problem. The limiting normalized queue-length 
process is denoted q(t). At the initial time moment t — there are k 
available client stations. Let T\, t 2 ,. . . be the moments of their 
breakdown, < T\ < r 2 < . . . < 7V The above moments of breakdown 
are associated with the behavior of the time dependent network, which 
can be considered as a network in semi-Markov environment. Let us 
then apply Theorems 15.41 and 16. 1[ (Last Theorem 16.11 is used conse- 
quently from one random interval to another. Here we do not explain 
the details of this application assuming that they are clear.) 

The random time interval [0, r*] is the lifetime of the entire system. 
Therefore q(t) is to be considered during the aforementioned random 
interval [0, r k ]. We assume that l = 1. Therefore, according to The- 
orem 15.41 and Theorem 16.11 we obtain that in the random interval [0, 
Tfc_i), q(t) = 0, while in the random interval [r^-i, Tfe) the equation for 
q(t) is 

(7.4) q(t) = (A - /i)(t - tv-O - A f ^ r(s)ds, 

Jo 

where r(t) is given by 

(7.5) r(t)= (l-£)(l-e- A *). 
In the last endpoint r k we set q(Tk) = 1. 
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We have the following relationships: 
(7.6) ¥{q(t) = 0} = g Q [1 - G(t)Y[G(t)} k -\ 

P{g(*) < 7 < 1} 

( 7 - 7 ) r, ^,m^A-1 



i=i 

where i 7 is such the value of t under which 
(7.8) (X-n)t-X [ r(s)ds = 7 . 



The value t 1 is found from the relation 



JT E-= 2 (■) [i - G(t - y] w - t 7 )] fc -dt 



and then from (17.81) one can find the corresponding value 7. 

If the value of 7 is not greater than a, then the value 9 of the interval 
[rfc_i, 9) is to be taken 9 = t^i + £ 7 . Otherwise, if 7 > a, then the 
value 9 is to be taken 9 = t&_i + t a . 

In the particular case of k = 2 we have the following results. Rela- 
tions (I7.6p . (17. 7p and (17.91) reduces correspondingly to 

(7.10) ¥{q(t) = 0} = [1 - G{t)}\ 

(7.11) F{q(t) < 7 < 1} = [1 - G(t)][l - G(t - t 7 )], 
and 

, 71 ^ / °°[l-G(t)][l-G(t-t 7 )]dt _ 
1 j / °°[l-G(t-t 7 )pdt 

7.3. Numerical calculation. We consider the following example for 
k = 2: A = 4, /i = 3, a = 0.2, P = 0.95, G(x) = 1 - e" 2x . From (17321 
we have: 

j Q OC e _ 2(t _ t7)e _ 2tdt 



e - 2t i = 0.95. 



Solution of the equation e' 21 -' = 0.95 yields t 7 = 0.025647. From (17781) 
we obtain: 



7 



fcy /-0. 025647 

e" 4 'dt = / e" 4 *dt = 0.25 - 0.25e-° 102588 « 0.024375. 



./0 
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This value of 7 is less than a = 0.2, and therefore this value 7 = 
0.024375 is a required value for parameter, which defines a desired 
confidence interval. 

8. Discussion of new problems and associated 
monotonicity conditions for the networks in markov 

environment 

Theorem 16.11 looks very complicated, and its further analysis is very 
difficult to make a conclusion on the behavior of queue-length processes. 
For example, it seems very difficult to obtain any numerical character- 
istics of normalized queue-length processes analytically, Mxj(<Ji At) for 
example. Therefore, the numerical work should be based on simulation 
of semi-Markov environment in order to obtain required performance 
characteristics of the process. (By simulation of semi-Markov environ- 
ment we mean a multiple realizations for a Markov process in order 
to calculate required numerical performance characteristics of the pro- 
cess.) 

For the purpose of performance analysis we also should restrict the 
class of networks and processes describing the behavior of queue-length 
processes in client stations. This restriction is related to application of 
the results rather than development of the theory. In many practical 
examples the quality characteristics of networks are changed monoton- 
ically resulting in one or other strategy of repair mentioned in Section 

m 

The aforementioned comparing of two different strategies for fixed in- 
terval (0, T) requires application of Theorem 16.11 and the problem can 
be solved without any additional assumption requiring monotonicity. 
However, under general settings we cannot answer to many significant 
questions. One of them is How behave this criteria when the considered 
time interval is changed? For example, we have two strategies corre- 
sponding two different initial conditions of semi-Markov environment, 
and suppose we concluded that the first strategy is more profitable 
than the second one for specific time interval (0, T). Is this conclu- 
sion remains correct (or becomes not correct) for another time interval 
(0, T*)? Another typical question is as follows. Again, we have two 
strategies corresponding two different initial conditions of semi-Markov 
environment. Suppose we established that for an interval (0, T) the 
both strategies are equivalent. Let T* be a new time instant, and 
T* > T. Which one of the strategies is now more profitable in the new 
time interval (0, T*), the first or second one? The same question can 
be asked under the opposite inequality T* < T. 
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These questions can be answered in the case when the class of the 
processes studied numerically has a monotone stricture and is described 
by the properties listed below. Then in certain cases the behavior of 
queue-length processes in client stations and consequently a conclusion 
about better strategy for other time intervals can be established as 
well. 

Assuming for simplicity that the environment is Markov, then afore- 
mentioned properties are as follows. 

(1) For any two positive integers I < m assume that z^ m > z m j. 
Recall that z^ m /\t + o(At) (I ^ m) are the transition probabilities 

from the state £i to the state £ m of a homogeneous Markov process for 
a small time interval (t, t + At). 

(2) \j(£i) < \j(£ m ) for all j = 1,2, ... ,k, and I < m. 

Property (1) means that the Markov process Z(t) is an increasing 
process in the following sense: for two time moments 0\^\ and 07 we 
have Z(ct/_i) < st Z{a{), which means that the state of a Markov process 
in time oi_\ is not greater (in stochastic sense) than that state in time 
Oi for any integer positive I. The above property remains correct for 
any ti < t 2 , i.e. Z{t x ) < st Z(t 2 ). 

Property (1) also means that for two Markov processes Z\{t) and 
Z2(t) having the same transition probabilities, but different initial con- 
ditions satisfying Zi(0) < st Z 2 (0), we also have Z\{t) < st Z 2 {t), t > 0. 
(For details of the proof of these properties see e.g. Kalmykov |20j.) 

Consequently, from property (2) we have Aj(£(cr/_i)) < st \j(£(o~i)) 
for all j = l,2,...,k and any integer positive /. Moreover, for all 
j = 1, 2, . . . , k and any ti < t 2 we have \j(£(ti)) < st Xj(£(t 2 )). 

Thus the rates Xj(£(t)) are increasing in time. As a result, the queue- 
length processes in client stations increase sharper than in the case of 
fixed Xj of "usual" network, and more extended problems mentioned 
in this section seem can be solved as well. We however do not provide 
their solutions in the present paper. 

In the next section, numerical investigation for concrete client server 
networks in a given Markov environment, satisfying the above two prop- 
erties is provided. 

9. Example of numerical study 

In this section we do not intend to challenge a problem of comparing 
two different strategies or finding an optimal strategy. We only show 
(step-by-step) how to study the behavior of queue-lengths in client 
stations numerically. However, the detailed explanations of the given 
example can help to solve some of the aforementioned problems of 
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P 



Section [2j The example models a Markov environment, i.e. in our 
example we simulate exponentially distributed random variables de- 
scribing the state changes in the Markov environment. (In the given 
case by simulation we mean one realization of the process in order to 
study numerically a specific sample path of normalized queue-length 
process.) 

We consider the simplest case of Markov transition matrix of the 
order 4 

/0 1 o\ 
10 
1 
\0 1/ 

associated with the continuous Markov process Z{t). In this matrix 
Pi,i+i — 1) I — 1)2,3, and P 44 =l. This means that if the initial 
state of the process is S (0) = Si, then the next state is S(<Ji) = S 2 - 
Consequently, S(a 2 ) = S 3 , and S(a 3 ) = £ 4 . Then S(ai) = £ 4 for all 
I > 3. Assume also that Zij+x = 1, / = 1, 2, 3, so that E(o"z — <7/_i) =1, 
1=1,2,3. 

For simplicity, the network contains only 2 client stations. Assume 
that Ai(£i) = l, Ai(£^2)=2, and for 1=3,4, Ai(£;)=3. We also assume 
that Xi(Si) = A 2 (£i) for I = 1, 2, 3, 4. The values /Ui = ^ — 2. Next, f3\ 
= P 2 = 0.1, i.e. at the initial time moment each client station contains 
10% of all units in the queue. 

We set T=3, and study behavior of queue- length processes in client 
station in the time interval (0, 3). By simulation we obtained the 
following exponentially distributed random variables: 0.5488, 1.0892 
and 1.8734. The sum of these 3 random variables is greater than 3, so 
this quantity of exponentially distributed random variables is enough 
for our experiment. 

Note, that Ai(0) = A 2 (0) = 0.8. There three time intervals: [0, 
0.5488), [0.5488, 1.6380), [1.6380, 3). 

For the time interval [0, 0.5488) the two client stations are abso- 



lutely non-bottleneck because 3 ^ =0.5, and we have the following 



equations: 

^ xi{t) = x 2 {t) = 0.1 - 0.96t - 0.8 jf z{s)ds, 

z(t) = -1.5 (1 - e- 1 ' 6 *) . 
Therefore, from ( 19. II) we obtain: 

(9.2) xi(t) = x 2 (t) = -0.65 + 0.24t + 0.75e- L6 '. 
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Substituting 0.5488 for t in ( 19. 2ft one can see that 

X!(0.5488) = x 2 (0.5488) « -0.2066. 

The endpoints are negative, therefore we are to find such the values T\ 
and r 2 such that X\{t\)=S$ and x 2 (r 2 )=0. In our case T\ — r% ~ 0.117. 
This means that q'i(t) = qii$) = for all £ > 0.117 of the given interval 
[0, 0.5488). Therefore, in the endpoint of this interval q± (0.5488) = 
g 2 (0.5488) = 0. 

Consider now the time interval [0.5488, 1.6380). In point 0.5488 we 
now set Xl (0.5488) = x 2 (0.5488)=0. Therefore, Ai(0.5488) =A 2 (0.5488) 
=2, and the both client stations are bottleneck in [0.5488, 1.6380). 
Since Aj (°- 5488 ) = i ; j — \ 2, then X\(t) and x 2 {t) are equal to zero in 

this interval, and qi(t) = g 2 (i) =0 in this interval as well. 

We arrive at the last time interval [1.6380, 3). Similarly to the above, 
we have x^l.6380) = z 2 (1.6380)=0, and Ai(1.6380) = A 2 (1.6380)=3, 
and the both client stations are bottleneck in [1.6380, 3). However, 
in the both client stations we have Aj( ' 1 ' 6380 ' > = 1.5, j = 1,2. Therefore, 
after a little algebra we have the following equations: 

q J (t)= 1 -(l-e-^~^), j = l,2, 

for all t from the interval [1.6380, 3). 

10. Concluding remarks 

In the present paper we introduced a class of client /server networks 
in order to study performance measures of real client /server networks. 
Our analysis was based on the results of earlier papers related to closed 
queueing networks with bottleneck. However, for purpose of real appli- 
cations, we developed the earlier results and provided complete analysis 
of standard bottleneck client /server networks. We then extended our 
results for client /server networks in semi-Markov environment. The 
results obtained in this paper are then used for analysis of confidence 
intervals of client /server networks with failing client stations. Numer- 
ical study given in this paper will help to clearly understand solution 
for many related problems. The future work can be related to appli- 
cation of the theoretical results of this paper to concrete technological 
problems similar to those formulated in Section [2J 
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